Fix RTL cursor coherence and Org table whitespace highlighting - #708

Problem

Follow-up to merged PR #707, addressing the RTL cursor review and CodeRabbit's unchanged-point replay nit. Upstream merge 03beb91e959ec9afe5c9d35f42b5b0b04843d262 has the same source tree as the reviewed/tested pre-change head a48137e1f7.

Problem and reproduction

Greptile's new comment (discussion_r4502358156) was reproducible. Several visually reordered replacement-string glyphs share buffer coverage. Cursor placement chose the smallest string index, while fresh vertical capture chose the first visually covering glyph. With logical-first Hebrew א at height 2.0 and visually earlier בג at height 0.5, the regression failed at cursor height/ascent 22/19 versus 24/19.

Additional review verified the same identity mismatch in replay width/paint. An expanded differently colored replacement failed whole-cursor equality: replay selected the visually first blue background instead of the logical cursor glyph. The original fresh capture retained the underlying buffer face. The new test checks the selected replacement's width and red cursor foreground and compares the entire fresh/replayed cursor.

CodeRabbit's unchanged-point nit was also reproduced: without font measurements, a second unchanged frame unnecessarily fell back to full layout despite an authoritative retained cursor. The new test failed with cursor_only_windows = 0, expected 1.

GNU Emacs reference, root cause, and solution

Pinned GNU Emacs 31.1, commit a360712c9d272d950d8d8255ef74570f7e90b7d9:

• src/xdisp.c, set_cursor_from_row: replacement selection uses the smallest string position within its occurrence.
• src/xterm.c, physical cursor drawing: the glyph from get_phys_cursor_glyph supplies physical geometry.
• src/xterm.c, x_set_cursor_gc: non-default cursor paint uses the selected glyph face's foreground/background.

The existing semantic slot resolver now carries both materialized column and text-glyph index. Fresh and replay paths share that selection for geometry, replacement width, and paint, rather than independently searching visual order. Selection happens before checking measurements, so an unmeasured chosen glyph cannot be silently replaced by another measured candidate. Hidden-text and row-end approximations retain their previous fallback contracts; explicit display captures and ordinary buffer styling remain unchanged.

Cursor-only replay skips reconstruction when it already has an authoritative retained cursor. Moved, unmeasured concrete-font points still decline replay and retry fresh with checkpoint restoration.

Original RTL-phase regression verification

• Both new engine regressions failed before the production correction; focused convergence checks: 9/9 passed.
• New native RTL probe: pinned GNU passed; the previous Neomacs executable failed (height 22 versus its 27-pixel row). It compares cold/moving-point physical cursors and requires actual cursor-only row reuse through the exact telemetry interval in Neomacs.
• Final complete layout target: 3,349 run; 3,317 passed; 32 failed. Failure names exactly match the preceding baseline: no new failures. The suite is not fully green.
• Final Neomacs GUI/control target: 14/14 passed; pinned GNU 31.1 target: 14/14 passed. Eleven tests launch native GUI probes (one runs four mixed-newline cases); three are pure controls. The new RTL probe proves cursor-only row reuse in Neomacs and geometry in GNU.
• Headerpad release compilation: passed (4m36s). Complete fresh-build pipeline: passed, including byte compilation and dump regeneration.
• Library Clippy, targeted Rust formatting, GNU Lisp parentheses, and patch-whitespace checks: passed (Clippy warnings remain).

Commands actually executed

cargo nextest run -p neomacs-layout-engine --no-fail-fast --test-threads 6
cargo clippy -p neomacs-layout-engine -p neomacs-gui-tests --lib
RUSTFLAGS='-C link-arg=-Wl,-headerpad_max_install_names' cargo build -p neomacs --features webview --release
RUSTFLAGS='-C link-arg=-Wl,-headerpad_max_install_names' cargo xtask fresh-build --release --skip-build
NEOMACS_GUI_TEST_BINARY="$PWD/target/release/neomacs" cargo nextest run -p neomacs-gui-tests --test gui -E 'test(commit_editing::)' --test-threads 1
NEOMACS_GUI_TEST_BINARY=/tmp/neomacs-gnu-emacs-31.1-reference/src/emacs cargo nextest run -p neomacs-gui-tests --test gui -E 'test(commit_editing::)' --test-threads 1

The final two commands ran sequentially after the complete pipeline. Source checks also ran rustfmt --check --edition 2024 --config skip_children=true on the four modified Rust files, pinned GNU check-parens on cursor-rtl.el, and git diff --check. Read-only review signed off after the width/paint correction. Temporary logs and the exact baseline comparison are under /tmp/neomacs-pr707-rtl/.

Limitations

Verification is on Apple Silicon macOS. Linux, Wayland, and Windows runtime execution was not performed. The original RTL phase rebuilt target/release/neomacs without repackaging apps. A separate Org-only test app was built subsequently; the combined executable/app was not rebuilt for this PR-only update. Automated re-review and approval are separate from local verification. The Org whitespace correction is now included and documented below. Missing-font fallback remains a separate, uncorrected compatibility issue.

Copilot missing-telemetry follow-up (6988f02e32)

The RTL fixture now reports Missing layout telemetry if Neomacs exposes its snapshot capability but the configured stats file has no attributes, rather than silently entering GNU-reference mode. Both replay boundary reads share the strict helper.

A new native negative control points at a missing parent directory. Before the fix, the fixture incorrectly exited successfully and the regression failed. Afterward it exits with the expected explicit diagnostic and never emits the GNU skip marker. On GNU the negative control explicitly simulates Neomacs capability; the ordinary RTL probe still uses GNU's genuine geometry-only path.

Actual checks: focused RTL/control target 2/2 passed on Neomacs and 2/2 on pinned GNU 31.1; complete Neomacs GUI/control target 15/15 passed. Targeted Rust formatting, pinned GNU Lisp check-parens, and git diff --check passed. This follow-up changes only the fixture and its GUI tests: no runtime rebuild or full engine-suite rerun was needed, and app bundles were not changed. Logs: /tmp/neomacs-pr708-telemetry/.

Org table whitespace correction (3400610f0e)

This PR now also includes the separately verified Org table fix, at the reporter's request. It is the identical patch previously tested as d5a219a7ed (stable patch IDs match), cherry-picked onto the existing RTL/telemetry branch. No personal font settings or missing-font workaround are included.

Problem/reproduction: with show-trailing-whitespace enabled, an org-modern table border displayed as (space :width (3)) was mistaken for trailing source whitespace. Interior padding and the closing border acquired the trailing-whitespace background. A synthetic table reproduced the reported coral band without opening a private document or executing SQL. The old native rendered-face test incorrectly highlighted rows (4 2 1 0); only the genuine trailing-space control on row 4 should highlight.

GNU reference: exact GNU Emacs 31.1 a360712c9d272d950d8d8255ef74570f7e90b7d9, src/xdisp.c:24835–24937 (trailing_whitespace_p, highlight_trailing_whitespace), and the point-sensitive reuse exclusions at 19880 and 21804. The last candidate must correspond to real source spaces/tabs ending at newline, carriage return, or accessible buffer end, with suppression when the endpoint is point. Source classification occurs once before backward refacing; excluding all displayed stretches would incorrectly lose valid highlighting. String-owned glyphs block the scan rather than behaving as nil-object synthetic padding.

Root cause/solution: the renderer classified glyph appearance instead of the buffer source and skipped string-owned glyphs too broadly. An execution-only borrowed source predicate now supplies the GNU classification to both newline paths, without copying the buffer or coupling the stored line-end plan to buffer storage. A second RED test exposed stale body-face reuse after point motion; cursor and scroll replay now decline that shortcut when point-dependent whitespace highlighting can change. Unchanged-point and ordinary non-whitespace-sensitive reuse remain eligible.

Regression evidence: the initial engine regressions failed before correction, positive space/tab controls remained covered, and the point-motion regression failed with retained highlights [1, 2, 5, 6, ...] versus fresh [5, 6, ...]. Focused source/whitespace/scroll checks passed 50/50. The Org-only full target ran 3,352 tests: 3,320 passed, the same 32 baseline failures. The independently built Org-only native/control targets passed 14/14 on Neomacs and 14/14 on pinned GNU; the signed test bundle's Org face assertion passed 1/1. GNU checks the equivalent source setup, not rendered face colors, because it lacks Neomacs's snapshot API. Its control is not a GPU color oracle.

Combined-branch verification: cargo nextest run -p neomacs-layout-engine --no-fail-fast --test-threads 12 ran 3,354 tests: 3,322 passed and 32 failed. The failure-name set exactly matches the preceding baseline: no new failures, but the suite is not fully green. Library Clippy, targeted Rust formatting, patch-whitespace checks, and pinned GNU Lisp parentheses checks passed. The combined fixture/control target passed 16/16 on pinned GNU. Prior native Neomacs component results above are historical; the combined Neomacs executable and app were not rebuilt for this PR-only update.

Limits: macOS ARM64 execution only; no new end-of-buffer-tail highlighting path is added. The separately packaged Org test app still contains the Org-only source, not the unmerged RTL changes. Original app bundles were not replaced. Missing-font fallback is a separately identified compatibility issue, not corrected or hidden by this patch. Temporary logs: /tmp/neomacs-org-whitespace/ and /tmp/neomacs-pr708-combined/.

Earlier review follow-up (1c766004ef, 5a5ef235ab)

Commits: 1c766004ef (point-dependent edit reuse) and 5a5ef235ab (cursor policy, placement and evidence controls).

Problem, reproduction and root cause

• Cursor-only reconstruction did not consistently apply stretch-width policy. Fresh capture also enlarged narrow displayed spaces rather than preserving their actual width. The narrow regression failed with replay width 3px versus fresh width 12px.
• RTL physical placement selected the correct string glyph, but the logical anchor kept a leading-edge integer override. The independent regression failed with logical x=12 versus physical x=22.
• Trailing-whitespace faces depend on point and real buffer endpoints. Edit reuse could retain a previously suppressed run, even with unchanged numeric point when an edit above shifted the row. The regression reproduced missing highlights at positions 10/11.

GNU Emacs reference and solution

The reference remains GNU Emacs 31.1, exact commit a360712c9d272d950d8d8255ef74570f7e90b7d9.

• src/xdisp.c:get_phys_cursor_geometry (around 2707–2748): non-stretching box/hbar width is capped at min(column width, glyph width); it must not enlarge narrow stretches. A shared width resolver now serves fresh, cursor-only and scroll paths, while explicit vertical bars retain their ink width.
• src/xdisp.c:set_cursor_from_row (around 19010–19101): replacement placement follows the selected string glyph. Logical anchors now resolve finalized placement and share the physical snapshot's integer-override acceptance/rounding rule, including cursor-type=nil; explicitly resolved string cursor candidates are preserved.
• src/xdisp.c:try_window_id (around 22653): show_trailing_whitespace causes GIVE_UP(11). Edit reuse now declines this point/source-dependent highlighting, rather than relying on numeric point movement alone. Ordinary non-highlighting edits still reuse rows.

Regression and review coverage

All three new regression tests failed before the fixes and pass afterward. Expanded controls cover narrow/wide stretches, enabled/disabled stretching, box/bar/hbar styles, RTL/LTR replacement placement, hidden cursors, and changed/unchanged numeric point on edits.

The RTL placement test independently asserts the minimum string-index glyph's visual column and pixel-prefix x, not merely fresh/replay equality. New native stretch controls require actual cursor-only/reused-row telemetry on Neomacs. Org evidence is explicitly distinguished as ORG-TABLE-RENDERED-FACES-PASS on Neomacs versus ORG-TABLE-REFERENCE-SOURCE-ONLY on GNU; the latter is not a rendered-color oracle. Added meaningful function documentation also addresses the review's documentation-coverage concern; automated coverage results are checked separately.

Actual local verification

• cargo nextest run -p neomacs-layout-engine --no-fail-fast --test-threads 6: 3,357 run, 3,325 passed, 32 failed. Exact failure-set comparison with the earlier combined baseline: same 32 failures, no new failures.
• Focused bug regressions: 3/3 passed; subsequently expanded cursor controls: 2/2 passed.
• cargo clippy -p neomacs-layout-engine -p neomacs-gui-tests --lib and --all-targets: passed, with existing warnings.
• Header-padded release build and cargo xtask fresh-build --release --skip-build: passed. The latter skipped only the already-completed Rust build; Lisp byte-compilation and dump regeneration were not skipped.
• cargo nextest run -p neomacs-gui-tests --test gui -E 'test(commit_editing::)' --test-threads 1: 17/17 passed on the rebuilt Neomacs executable, and 17/17 passed with NEOMACS_GUI_TEST_BINARY pointing to the pinned GNU executable. These are GUI/control tests, including pure controls and the explicitly source-only GNU Org check.
• Targeted Rust formatting, patch-whitespace checks and pinned GNU Lisp parentheses checks: passed. Read-only patch review: no must-fix findings.

Logs: /tmp/neomacs-pr708-loop/. Execution was on macOS ARM64; Linux, Windows and Wayland were not executed. Existing application bundles were not replaced or repackaged; only the source-tree executable was rebuilt.

CI status

Local verification is not a substitute for upstream CI. GitHub build/test/lint/CodeQL workflows require maintainer approval for the fork; approval and the subsequent run results are tracked separately. Green review-bot checks alone are not treated as a passing build. Existing review threads have evidence-linked fix replies; fresh review feedback is being rechecked.

Earlier logical-cursor dimensions follow-up (ce5965e950)

Latest reviewed source: ce5965e9504efd27dd44904bc387d9e40b727e76.

Problem and reproduction. The new review finding was reproduced with áx\n, an RTL אבג display replacement on x, and cursor-type set to nil. The selected replacement glyph was 7 pixels wide, but querying the published CursorBottom anchor returned width 0: the canonical source-position lookup found the preceding combining character. That zero-width source point is legitimate and must not be reordered, discarded or enlarged.

GNU reference. GNU Emacs 31.1, commit a360712c9d272d950d8d8255ef74570f7e90b7d9: src/dispnew.c:buffer_posn_from_coords obtains ordinary-glyph width and height from the selected glyph (pixel_width, ascent + descent) and permits zero-width results; src/xdisp.c:set_cursor_from_row preserves replacement-string selection identity. This establishes the selected-glyph contract; it is not a claim that GNU exposes Neomacs's CursorBottom API.

Root cause and solution. Logical position was finalized after bidi selection, but its selected glyph's extent was lost. The consumer could consequently combine the replacement's x with another source point's dimensions. A typed, optional LogicalCursorDimensions now travels atomically with logical position in the output snapshot. Placement and measured extent come from the same slot-selection result. Physical cursor geometry remains first priority, explicit logical extent second, and the unchanged canonical source lookup remains the legacy/unknown-measurement fallback. No fake physical cursor is created for hidden cursors; legitimate zero-width extents remain representable.

Explicit string-cursor candidates retain their own emitted source-slot measurements, including mixed-font/bidi strings; window-default height is not substituted. Composition members without their own primitive do not invent measurements. Snapshot cloning preserves the extent, while all four cursor-clearing/slicing paths clear it. Five existing snapshot fixture initializers received only the new optional None field; their assertions were retained.

Regression tests.

• hidden_rtl_cursor_anchor_uses_selected_glyph_dimensions: failed before the fix (0 versus 7 pixels), then passed through real frame presentation/anchor APIs without a physical cursor.
• hidden_explicit_string_cursor_anchor_uses_selected_glyph_height: guards explicit candidates, including a middle string index that is not generic replacement index zero. A development-time default-height error was reproduced (13 versus measured 24) and corrected before publication.
• Core presentation tests cover flat, eager and deferred positions, physical-cursor priority, explicit zero width, absent cursor position, unchanged canonical combining-character points and no forced materialization.
• The stretch-policy test now independently asserts the configured two-pixel bar width, not only fresh/replay equality. Added documentation explains the selected-slot and measurement invariants.

Verification completed so far.

• Layout suite: 3,359 run, 3,327 passed, 32 failed. The failure set is exactly the same 32 recorded before this follow-up; no new failures. This is not an all-green suite.

• Core window tests: 224/224 passed.

• Production-library Clippy and layout/GUI all-targets Clippy: passed with existing warnings.

• Release build with macOS header padding: passed in 6m21s.

• Targeted Rust formatting and git diff --check: passed.

• Complete cargo xtask fresh-build --release --skip-build: passed; byte-compilation and matching dump generation were not skipped (the release executable was built immediately beforehand).

• Native GUI/control target: 17/17 passed on Neomacs, 17/17 passed on the exact pinned GNU reference. GNU's Org case is explicitly source-setup-only, not a rendered-face oracle; Neomacs checks rendered faces and actual replay telemetry where applicable.

• All five existing mouse/input tests whose snapshot initializers needed the new optional field: 5/5 passed.

Limitations. Local runtime verification is macOS ARM64; Linux/Windows execution remains unverified. Existing application bundles have not been replaced. Upstream GitHub Actions still require maintainer approval; the existing approval request is #708 (comment). Bot check success is not build/test CI success.

Earlier follow-up (c05687225b): finalized logical columns, single-primitive extents and hidden replay

Problem, reproduction and root cause

The next CodeRabbit review identified three independently reproduced gaps:

• A finalized RTL replacement moved the logical pixel position but left logical_cursor.col at the captured buffer column. The regression failed with column 1 instead of 3.
• An explicit cursor on (propertize (string 1) 'cursor 1) used the aggregate advance of ^A, paired with the height of just ^. With ctl-arrow enabled, the cursor anchor was 24px wide instead of the selected primitive's 12px.
• A second, unchanged redisplay of a hidden explicit string cursor discarded its logical position and extent: None instead of Some((0, 0, 0, 0)). Retained cursor authority required a physical cursor, while publication was paint/style-gated.

Minimal state for the explicit-string cases is a selected "a\n" buffer, point 1, cursor-type nil, and an overlay before-string carrying an integer cursor property. The Rust regressions query the actual retained snapshot and public VisualAnchor::CursorBottom geometry; they do not simulate the result by manufacturing a physical cursor.

GNU Emacs reference

Pinned GNU Emacs 31.1: a360712c9d272d950d8d8255ef74570f7e90b7d9.

• src/xdisp.c, set_cursor_from_row: integer string cursor properties select an actual glyph and override the ordinary buffer-position match; selected glyph position is recorded as both w->cursor.hpos and w->cursor.x, independently of whether the cursor is painted.
• src/dispnew.c, buffer_posn_from_coords: ordinary glyph bounds use that glyph's pixel width and ascent + descent, not all primitives emitted for one source character. Zero width remains valid.
• src/textprop.c, modify_text_properties and Fput_text_property: string interval mutation is distinct from buffer mutation; buffer modification hooks/ticks cannot certify an independently mutable display string. A pinned GNU control actually moved cursor from X to Y and printed (1 nil) then (nil 1) (pure string semantics, not a rendered-cursor oracle).
• src/xdisp.c, get_phys_cursor_geometry: a non-stretched stretch cursor is capped by the canonical frame column width. The added subpixel/one-pixel unit assertion checks existing Neomacs paint policy; it is not presented as a universally verified GNU zero-width rule.

GNU has no directly corresponding Rust CursorBottom API. The new hidden-anchor tests therefore establish Neomacs snapshot/presentation invariants, supported by the pinned source contract—not a fabricated cross-engine API oracle.

Solution and design

• Finalize logical x, output-space column and optional dimensions from the same selected placement. Display-space columns include structural prefixes; the line-number control proves those are not copied into logical output columns.

• Store paired dimensions from the same actual primitive on each source slot, leaving aggregate slot advance separate. All four emission paths preserve this identity; retained primitives are measured after item layout can change their extents. Unknown/unemitted composition-member measurements remain absent, and real zero widths are retained.

• Retain accepted logical position and optional extent independently of optional paint authority. Copy this pair only for unchanged point under the existing placement-identical replay key, before any style gate. No old logical coordinates are copied across a scroll.

• For moved/translated ordinary hidden selections, use a live source-absence proof: the complete text-property table is empty, effective default text properties are absent, overlays are absent, and local string prefixes are absent. This excludes category/alias indirection too, costs O(1), and does not confuse synthetic/control primitives with absence of a string source. Unknown/projected views default to no proof. Finalized ordinary selection is then published without manufacturing paint or requiring font measurements; unknown dimensions stay absent. Other unproven hidden selections retry a source walk at shared cursor-only/partial-walk boundaries before sealing output.

• A second review caught a real stale-dependency failure: mutating an existing before-string's cursor intervals without moving point or replacing the overlay retained x=0 instead of fresh x=25. A separate default-property regression retained dimensions Some(12×13) instead of fresh None. Both were reproduced RED before correcting their causes.

• Canonical producer walks now record the original contents and interval projections of all projected source strings, including property-less competitors, before candidate filtering. Original bounded CollectionReads certificates and immediate evaluator-owned roots are kept in an owner-only heap history sidecar, not numeric snapshots, worker rows or TLS-embedded retained matrices. An unchanged replay carries the original certificate; it never blesses skipped work by capturing an empty new dependency set. Unknown/incomplete or expired observations require a source walk. Nonintersecting temporary-string mutation retains the fast path in both regression setups.

• A weak OwnedRootRetainer registers original operands immediately without holding borrowed Context pointers or truncating anyone's scratch-root stack. Existing GC lease tracing/pruning semantics remain; a new exact-GC test verifies graph retention, release and declining registration after Context retirement. The original root-registry slot size/alignment—and therefore pinned Context ABI—is preserved with explicit reserved words and compile-time assertions. Closed frame histories release their leases on the next accepted frame commit.

• Revalidate the whole accepted window set after the final Lisp-capable boundary, not only each leaf at completion: a later leaf's fontification can mutate an earlier leaf's source string. The two-window regression actually failed x=0 versus fresh x=12 before the frame-level logical-input retry was added.

• Preserve original dependency authority across partial walks when fresh source identities are already in the original domain. If retained rows had no string inputs, actual newly walked inputs form a complete new certificate. Otherwise, an expanded source domain uses a frame-only union of original and fresh witnesses, and requires the next canonical walk; this bounds retained roots/history without rebasing skipped inputs. The unchanged edit-sync typing test exposed the overly conservative alternative and now remains intact.

Broader runs caught two ordinary hidden-reuse regressions and one decorated edit-reuse regression. The complete-source proof and bounded partial-domain transport correct them without weakening the existing tests. They deliberately avoid caching “this mutable string has no cursor property” under buffer/overlay ticks.

Regression coverage

• replacement_logical_cursor_tracks_the_selected_visual_slot: independent selected-column and pixel-prefix oracle; RTL/LTR, visible/hidden and line-number prefixes.

• hidden_explicit_string_cursor_anchor_uses_selected_glyph_dimensions: single glyph, multi-primitive control character and mixed-face/bidi explicit cursor on the middle string character; independently selected primitive width and height.

• hidden_explicit_cursor_survives_unchanged_replay: position + extent + actual public anchor preserved without painting; both visible and hidden branches require cursor-only reuse.

• hidden_explicit_cursor_reselects_after_point_and_start_changes: integer overrides survive point movement and translated retained rows through conservative retry; final geometry and emitted body primitives match a cold layout.

• stretch_cursor_only_replay_matches_fresh_width_policy: independent box/hbar/bar expectations for .01/.25/5-column stretches, both stretch settings; fresh and replay results each checked against raw selected glyph width and canonical (frame-char-width).

• hidden_canonical_cursor_replay_matches_fresh_positions: ordinary moved hidden selections and an empty final line match cold logical position/extent without requiring paint metrics.

• layout_property_absence_proof_is_complete_and_snapshot_stable: unknown property names invalidate the absence proof; a frozen empty snapshot remains empty after buffer mutation, across implemented text backends.

• Existing cursor_only_replay_preserves_hidden_buffer_cursor and point_only_automatic_scroll_reuses_retained_rows: unchanged tests preserve ordinary hidden cursor-only/navigation and scroll reuse.

• hidden_explicit_cursor_reselects_after_string_property_mutation: existing cursor moves X→Y, and a completely property-less competing string gains cursor; both compare with cold layout. Unrelated temporary-string mutation must preserve cursor-only reuse first.

• hidden_default_property_cursor_is_not_canonical: global defaults can introduce an explicit string cursor despite an empty raw property table.

• hidden_cursor_sources_are_revalidated_after_later_window_callbacks: a second leaf's one-shot fontification callback mutates the first leaf's string after its leaf-local validation. Actual RED was x=0 instead of fresh x=12; GREEN follows final accepted-window certificate validation after the final Lisp-capable boundary, before sealing. A changed certificate requests the existing logical-input frame retry.

• partial_source_domains_preserve_original_authority_and_bound_history: original Rc identity is preserved for known domains; empty domains admit actually observed new sources; expanded domains cannot chain. Mutation of either union arm must invalidate it.

• Existing typing_in_a_line_with_a_display_string_walks_only_its_rows remains unchanged and requires edit reuse after adding a display string.

• Core weak_retainer_registers_live_graphs_without_extending_context_lifetime, plus the unchanged existing owned-root GC tests.

The original three failures, both subsequent dependency/default failures, the later-window callback failure and the three broader reuse regressions were observed before their production corrections. Existing assertions were preserved or strengthened with independent expected values. Final command results are recorded below.

Verification

Actual final-source results on macOS ARM64:

• cargo nextest run -p neomacs-layout-engine --no-fail-fast: 3367 run, 3335 passed, 32 failed (288.501s). The exact failure-name set equals the published ce5965e950 baseline: zero new failures, none hidden or removed. This is not an all-green suite. The complete suite was also re-executed on exact published head c05687225b, after full Lisp/dump regeneration: the same 3367 / 3335 / 32 (272.389s), with an identical failure-name set.
• Targeted nextest selection covering the 14 regressions/controls above: 14/14 passed (9.452s).
• cargo nextest run -p neovm-core -E 'test(owned_roots::tests::)': 3/3 passed (0.501s), including exact GC retention/release and Context retirement. Linux/x86_64 every_context_offset_is_mains is cfg-excluded on this macOS host; it was not executed. The registry slot size/alignment assertions did compile here, and no layout-test constants were changed.
• cargo clippy -p neomacs-layout-engine -p neomacs-gui-tests --all-targets: exit 0, existing warnings remain.
• Changed-file rustfmt --edition 2024 --config skip_children=true --check and git diff --check: passed. An initial shell lacked Cargo's PATH; rerunning with $HOME/.cargo/bin exported passed.
• RUSTFLAGS='-C link-arg=-Wl,-headerpad_max_install_names' cargo build -p neomacs --features webview --release: exit 0 (4m46s).
• Same RUSTFLAGS, cargo xtask fresh-build --release --skip-build: exit 0, including full Lisp byte compilation and dump regeneration; no --no-byte-compile shortcut. --skip-build reused the immediately preceding Rust build.
• NEOMACS_GUI_TEST_BINARY="$PWD/target/release/neomacs" cargo nextest run -p neomacs-gui-tests --test gui -E 'test(commit_editing::)' --test-threads 1: 17/17 passed (29.462s), after the fresh pipeline.
• Same GUI command with the verified pinned GNU 31.1 executable: 17/17 passed (27.074s). GNU Org coverage is explicitly source-only, not a rendered-face oracle.

Intermediate runs with overly conservative guards or incomplete partial-domain transport exposed the three additional reuse regressions described above. Those runs/builds were superseded, not represented as passing final verification. No existing assertions were deleted or weakened.

Limitations

This is macOS ARM64 verification, not Windows/Linux/Wayland runtime verification. The GUI/control group includes pure selector/proof tests; its GNU Org case explicitly reports source-only evidence rather than rendered-face assertions. App bundles (dist/neomacs.app, the Org-only test app and /Applications) are not repackaged or replaced. Personal font settings and the previously documented unavailable-family fallback metadata defect are unchanged. Hidden moved/translated selections with decorated or otherwise unproven sources may conservatively use a full walk; the new certificate covers actual source-string contents/interval projections and original owner identity, not arbitrary nested mutable face/callback graphs or worker-produced cursor provenance. Missing certificates remain unknown.

Upstream Actions require maintainer approval for fork runs. Bot check success is not evidence that test/lint/build/CodeQL executed. The existing approval request is not reposted.

Earlier root-registration performance follow-up (ba5fc2fc50)

Problem / reproduction. string_properties registered a new lease for every projection, and each registration scanned all earlier leases. Deterministic regression tests reproduced 523,776 weak-lease inspections for 1,024 live registrations, and 100 leases for 100 projections of one original string. These are measured operation counts, not end-to-end layout speed claims.

Solution. Root each original source identity once, immediately before its first projection; continue capturing contents/interval observations on every projection. Registration reclaims an expired suffix cheaply, then performs a full sweep only at capacity. If more than half remain live, it reserves the old capacity as additional headroom, avoiding repeated full scans when a sweep frees only one slot. Each suffix entry is popped once; capacity sweeps have amortized-linear work. GC still traces only live leases and removes expired ones. Non-suffix dead native weak entries may remain until a capacity sweep or GC; they do not keep Lisp graphs live. There is no new epoch, layout field, Context ABI change, or weakened existing test.

Regression tests. repeated_string_projections_share_one_root_lease failed 100 vs 1 before correction. root_registration_pruning_work_is_amortized_linear failed with the operation count above; it covers growing live, long-lived-plus-transient and rotating non-tail sets and verifies exact GC releases the graph. The existing retired_leases_do_not_accumulate_between_collections initially caught an overly-lazy intermediate version (2 vs 1); expired-suffix reclamation preserves this test unchanged, rather than relaxing its assertion. Existing lease/graph/retainer lifetime assertions are retained.

GNU reference / compatibility. This is internal native root-registration bookkeeping; no new GNU behavior is inferred. The pinned GNU 31.1 source contract and prior source/GUI qualifications remain as documented above.

Verification. cargo nextest run -p neovm-core -E 'test(owned_roots::tests::)' --success-output immediate: 4/4 passed (0.719s), including the unchanged retired-lease test. Actual inspection counts: growing set 2,043 (previously 523,776), plus 9,215 for 4,096 transient registrations alongside 1,024 live leases, and 22,528 for 8,192 rotating non-tail registrations alongside 2,048 live leases. cargo nextest run -p neomacs-layout-engine --no-fail-fast: 3,368 run, 3,336 passed, the identical 32 baseline failures, no new failures (328.334s). Affected layout/GUI all-target Clippy, core-library Clippy, changed-file rustfmt and diff checks passed, with existing warnings. Headerpad release compilation passed. The complete fresh pipeline passed (no byte-compilation skip); freshly stamped native Neomacs GUI/control group 17/17 passed (29.285s), pinned GNU 31.1 group 17/17 passed (27.177s). GNU Org evidence remains source-only. A complete post-fresh rerun of the same sources also produced 3368 / 3336 / the identical 32 baseline failures (280.105s), with no new failure names. Intermediate failures/superseded builds are not passes. Logs: /tmp/neomacs-pr708-loop/round4b-* and round4c-*.

Limitations. macOS ARM64 only; Linux-only Context-offset execution remains unavailable here. The first native attempt reused the preceding dump after ordinary Cargo rebuilding and failed at startup with the executable's unstamped pdump fingerprint (2 pure controls passed; no GUI runtime proof). Although Lisp sources and dump/Context formats are unchanged, ordinary Cargo rebuilding restores a fingerprint placeholder. The complete cargo xtask fresh-build --release --skip-build pipeline was then run successfully, including byte-compilation/dump generation and executable fingerprint patching; the native retry results are above. App bundles and personal fonts remain untouched. Upstream build/test/lint/CodeQL require maintainer fork-workflow approval, requested previously; bot/label successes are not passing Actions CI.

Earlier source-object identity / GC safety correction (1d657bc7ab)

Problem / root cause. Greptile correctly identified that Value's PartialEq/Hash are structural, not allocation identity (crates/neovm-core/src/tagged/value.rs:266–289). Equal-text strings can have different intervals and lifetimes. Using FxHashSet<Value> both for deduplicating roots and for partial-domain subset checks could omit a distinct object's root or discard a newly observed object's certificate as an apparently known domain. This corrects the earlier mistaken identity assumption; it does not change Lisp equality.

Solution. Both sets now use a private SourceStringIdentity(usize) from value.bits(). Its derived equality/hash are over the original tagged allocation identity, never mutable contents. Original evaluator-owned leases remain registered before first projection and live with the certificate; all contents/interval observations and original-world validation remain unchanged. The same identity rule governs partial-domain expansion, so an equal-text new allocation gets the bounded frame-only union rather than reusing the old object’s authority. No Context/dump layout, public equality, global epoch, or worker snapshot changes.

GNU reference. Pinned GNU 31.1 a360712c9d272d950d8d8255ef74570f7e90b7d9, src/fns.c:2803–2814 (Fequal) and 3006–3012 (internal_equal, strings): ordinary equal compares bytes/length and ignores string intervals. A batch control with separately allocated equal-text strings and cursor on only the second returned (nil t 1) for (eq a b), (equal a b), and its cursor property. This supports keeping Lisp equality intact; native certificate identity is a separate contract, tested with actual Neomacs GC, not a claimed GNU certificate API.

Actual regression proof. equal_content_sources_keep_distinct_roots creates distinct equal-text strings with different properties, measures their complete live graph under temporary setup roots, drops those setup roots before exact GC, and checks certificate-only survival, then final release. Structural-key semantics reproduced 3830 surviving objects vs the complete graph's 3833 (no unsafe dereference of reclaimed values). equal_content_new_source_expands_partial_domain also failed because the different allocation was falsely replayable. Initial harness compilation/graph-size assumptions were corrected before the faithful RED proof (round5-identity-red4.log). Both corrected regressions and the two existing source-certificate tests passed 4/4 in round5-identity-green2.log (0.371s); existing assertions were not weakened.

Verification. cargo nextest run -p neomacs-layout-engine --no-fail-fast: 3370 run / 3338 passed / the identical 32 baseline failures, zero new failure names (286.341s). This is not an all-green suite. Headerpad release compilation, complete cargo xtask fresh-build --release --skip-build (including byte-compilation/dump/fingerprint generation), affected layout/GUI all-target Clippy and changed-file formatting/diff checks passed; existing warnings remain. Fresh native Neomacs GUI/control group 17/17 passed (29.375s), pinned GNU group 17/17 passed (27.350s); GNU Org evidence remains source-only. Read-only review found no additional issues in the two-file delta; it did not independently rerun tests. Logs: /tmp/neomacs-pr708-loop/round5-*.

Limitations. Runtime execution remains macOS ARM64-only. GNU Org rendering is still source-only evidence; the separate missing-family metadata fallback remains unfixed. No application bundles or personal font settings changed. Upstream Actions remain subject to previously requested maintainer approval; bot/label successes are not executed build/test CI.

Final bounded review-follow-up batch (e4fe564e2f)

The contributor asked to finish this batch and then stop. No further manual review request or review-watching round will be started after this push.

Problem, reproduction and root causes

Seven findings were established and addressed:

• 4504495569 — visible string cursor override: moving point in ab\n with a before-string X carrying integer cursor coverage lost the semantic selection for a visible cursor. The regression reported logical None versus the cold layout's Some((0, 0, 0, 0)). Reconstructing ordinary buffer point cannot replace an explicit string override, regardless of paint visibility.
• 4504495580 — mutable face payload: a retained before-string's original face plist could change via cons mutation without changing the string's interval identity. The regression retained extent 25×24 where the cold layout produced 1×2. Capturing only the string-table getter missed the actual face decoder's cons reads.
• 4504495586 — finalized composite extent: Arabic contextual composition widened the emitted composite from 8 to 32 after its selected-primitive metrics had been copied. Both builder regressions failed 8 versus 32 before the correction.
• 4504495591 — deleted owner roots: idle maintenance retained a deleted frame's source lease until another presentation was accepted. After separately accounting for ordinary frame-parameter reclamation, exact GC retained 4043 objects instead of 4042. This reproduction does not assume deleting a frame leaves the rest of its Lisp graph unchanged.
• 4504495593 — historical/prepared authority: cached rows needed their own original source authority, not the most recently accepted viewport's certificate. Missing-authority unit coverage failed before admission was corrected; additional tests validate original A after a distinct newer B capture and preserve original computed-page reads. This is producer/certificate unit evidence, not a claimed native GUI A→B oracle.
• 4504495596 — local window-end rollback: the late source rejection returned a retry without consuming its speculative window-end token. The isolated regression failed rejected local token must be consumed, using a real changed source observation and a different speculative end record.
• 4504495605 — unknown observation admission: eight real nested collection-read scopes forced incomplete observation. The old path still accepted a frame. Starting without an accepted presentation, the regression now requires bounded abort, no prepared/active presentation, preservation of the prior window end, and successful recovery outside the exhausted capture scope.

4504495573 remains unconfirmed: neither the forward hidden scroll case nor hidden edit cases with dirty rows above/below the reused point row reproduced the alleged stale logical cursor. These tests require actual scroll/edit reuse and nonzero reused rows, no physical cursor, and equality of logical position and extent against a cold layout. No speculative production change or claim of a reproduced failure is made for this finding. Its thread remains open with the attempted reproduction and limitation documented. The requested reused-row test coverage was added.

GNU Emacs reference

Reference checkout: GNU Emacs 31.1, a360712c9d272d950d8d8255ef74570f7e90b7d9.

• src/xdisp.c:18775–18808: integer string cursor coverage is checked before ordinary point matching.
• src/xdisp.c:34250–34283, 32284–32313: completed composite dimensions are established before the composite glyph is appended.
• src/xdisp.c:21740–21759: window-end validity is not published as a completed redisplay result prematurely.
• src/xfaces.c:2700–2810, merge_face_ref: native face realization reads the actual plist's cons keys and values. The batch control mutating a face plist shared by a string prints (:height 0.1).

Neomacs's collection journals, source certificates, and VisualAnchor::CursorBottom are internal mechanisms, not GNU APIs. The face-plist batch control is not a GNU GUI redisplay-invalidation oracle.

Solution and design

• Explicit semantic selection is required for both painting and nonpainting cursors. To preserve existing ordinary replay, a private typed proof certifies observed absence of string cursor overrides on the ORIGINAL source inputs; it is refreshed only after source validation at cursor-only consumption. Category-dependent selection stays conservative. Successful owned row programs already reject string cursor properties. This proof is not transferred to partially changed source walks.
• Root the original face operand before its callback-free native decode and record the actual reads, including cache dependency propagation. Decoded operand allocation identities participate in partial-domain inclusion. No reconstructed plist substitutes for the original graph, and unrelated mutations do not blanket-invalidate reuse.
• Refresh only previously measurable source-slot primitive dimensions after an append completes. Preserve intentional None composition-member metrics and the first primitive of multi-primitive control characters; do not substitute aggregate source-slot advance or a whole-string glyph checkpoint.
• Retire dead frame histories and prepared entries at idle and frame-attempt boundaries, independently of accepted presentations.
• Carry each selected historical/prepared row plan's ORIGINAL authority through the per-attempt ledger, revalidate after callbacks and preserve it on acceptance. Computed rows keep their original collection reads and leases. Certificate unions are flattened, identity-deduplicated and bounded to 32 leaves; they do not form growing historical chains. The existing single-page coverage/remapping/face-admission algorithm is unchanged.
• Consume and reject the local window-end token before requesting a source retry. Every finished leaf must provide a complete source certificate; otherwise use the existing bounded retry/abort coordinator, never a fabricated empty certificate.

No Context/retained-matrix ABI changes, worker Lisp roots, global invalidation epoch, personal font settings, or application-bundle replacement were introduced in this batch.

Regression coverage

New tests cover both cursor styles, face cons mutations and unrelated writes, original operand/domain preservation across ancestor mutation and GC, ordinary/explicit/category cursor classification, bounded/deduplicated certificate unions, finalized composite metrics and control-character guards, dead-frame GC without a presentation, original historical/computed evidence, speculative-end rollback, exhausted observation scopes, and real hidden scroll/edit reuse versus cold layouts. Existing replay assertions and tests were retained. Five old prepared-row test tuple destructures changed mechanically for the added certificate return value; their assertions were unchanged.

Verification

Executed on macOS ARM64:

• cargo nextest run -p neomacs-layout-engine --no-fail-fast: 3391 run, 3359 passed, 32 failed (288.077 s). Failure names are EXACTLY the 32 from the preceding published 1d657bc7ab run; zero new failures, not an all-green suite.
• Focused group excluding the already established tall-face baseline fixture: 61/61 passed (5.200 s). The two final source-proof/union tests were subsequently included and passed in the complete 3391-test run.
• cargo nextest run -p neovm-core -E 'test(owned_roots)': 4/4 passed (0.651 s), retaining GC/lifetime and amortization assertions.
• cargo clippy -p neomacs-layout-engine -p neomacs-gui-tests --all-targets: exit 0, existing warnings remain.
• RUSTFLAGS='-C link-arg=-Wl,-headerpad_max_install_names' cargo build -p neomacs --features webview --release: exit 0 (4m43s).
• Changed-file Rustfmt check and git diff --check: exit 0.
• GNU 31.1 shared-face-plist mutation batch control: exit 0; actual source HEAD rechecked against the pinned commit.
• RUSTFLAGS='-C link-arg=-Wl,-headerpad_max_install_names' cargo xtask fresh-build --release --skip-build: exit 0, full byte-compilation/dump/fingerprint pipeline (no byte-compilation skip).
• NEOMACS_GUI_TEST_BINARY="$PWD/target/release/neomacs" cargo nextest run -p neomacs-gui-tests --test gui --no-fail-fast -E 'test(commit_editing::)' --test-threads 1: 17/17 passed (29.290 s).
• The same GUI/control group with the pinned GNU executable: 17/17 passed (27.237 s). The GNU Org case remains explicitly SOURCE-ONLY, not a rendered-face oracle; these are GUI/control groups, not 17 independent GPU image oracles.
• No generated source Lisp changes resulted. App bundles were not repackaged or installed.

The complete Rust layout suite above ran before the final fresh regeneration, against the existing generated Lisp tree. The source Lisp tree is unchanged. No exact-published-head post-fresh full-suite rerun is claimed for this batch.

Limitations and stop boundary

• The reused-row stale-anchor concern above could not be reproduced and remains open.
• Composite metric refresh is append-local. Previously returned progress is not retroactively rewritten if a separate later append changes an earlier composite; no end-to-end cross-append reproducer was established in this batch.
• Upstream test/lint/build/CodeQL workflows previously required fork approval with zero jobs. The existing approval request was made once; it is not being reposted. Bot success is not a passed build.
• No further manual review request or monitoring round will be started after delivery, as requested. New automated comments may still arrive independently.
• Runtime checks here are macOS ARM64 only. Linux/Windows and Linux-only Context-offset tests were not executed. Existing separate missing-family font fallback metadata behavior remains outside this batch.